build: Test Models Against the Full Docker Stack #3183
Conversation
allgandalf
force-pushed
the
Docker-Stack-Testing
branch
from
92ff4ac
to
f27780a
Compare
| @@ -300,6 +300,7 @@ services: | |||
| api: | |||
| image: pecan/api:${PECAN_VERSION:-latest} | |||
| user: "${UID:-1001}:${GID:-1001}" | |||
| restart: unless-stopped | |||
There was a problem hiding this comment.
It can be useful to automatically restart the api container if it ever fails when we bring up the whole docker stack
| sudo apt-get -y install docker-compose | ||
| - name: Build and run containers | ||
| run: | | ||
| docker system prune --all --force |
There was a problem hiding this comment.
Why is a prune needed here? Won't it be running in a fresh VM (presumably with ~empty docker cache) every time?
| container: | ||
| image: pecan/depends |
There was a problem hiding this comment.
I'm rusty on the details here -- Does running this test inside the depends container mean these deps are used directly, or do we wind up pulling the whole container again to be run inside itself? Would it work to not specify a container at all let the Docker stack run directly on the host?
| docker-compose ps | ||
| sleep 20 # Adjust the duration as needed | ||
| docker-compose ps | ||
| sleep 20 | ||
| docker-compose ps | ||
| sleep 20 |
There was a problem hiding this comment.
This should probably be a loop over an actual status check; maybe something like while [[ $(docker compose ps --format '{{.State}}'|sort|uniq) != "running" ]]; do sleep 10; done
| - name: Display running containers | ||
| run: | | ||
| docker-compose ps | ||
| docker network ls | ||
| docker network inspect bridge |
There was a problem hiding this comment.
How useful is this step for unattended CI run? Will these outputs be helpful for debugging or will they just be filling the log up?
| -F 'notes=' \ | ||
| 'http://172.17.0.1/pecan/04-runpecan.php' | ||
|
|
||
| sleep 200 |
There was a problem hiding this comment.
As above, probably want to loop over a status check and sleep rather than assume a set duration
| # ----------------------------------------------- | ||
| # cURL Request Template | ||
| #----------------------------------------------- | ||
|
|
||
| # Be sure to use the -H header and also use the bridge IP address of the Docker host (172.17.0.1) and not localhost | ||
|
|
||
|
|
||
| # curl -v -L -X POST -H "Host: pecan.localhost" \ | ||
| # -F 'hostname=' \ | ||
| # -F 'modelid=' \ | ||
| # -F 'sitegroupid=' \ | ||
| # -F 'siteid=' \ | ||
| # -F 'sitename=' \ | ||
| # -F 'pft[]=' \ | ||
| # -F 'start=' \ | ||
| # -F 'end=' \ | ||
| # -F 'input_met=' \ | ||
| # -F 'email=' \ | ||
| # -F 'notes=' \ | ||
| # 'http://172.17.0.1/pecan/04-runpecan.php' | ||
|
|
||
| # Adjust the sleep time according to the model you are running | ||
|
|
||
| # Append the workflowid and ID after `out/` by 1 for every consecutive run | ||
|
|
||
| # curl -v -L -H "Host: pecan.localhost" \ | ||
| # 'http://172.17.0.1/pecan/dataset.php?workflowid=(ID)&type=file&name=out/(ID)/sipnet.out' |
There was a problem hiding this comment.
These lines seem redundant with the ones above
| # ----------------------------------------------- | |
| # cURL Request Template | |
| #----------------------------------------------- | |
| # Be sure to use the -H header and also use the bridge IP address of the Docker host (172.17.0.1) and not localhost | |
| # curl -v -L -X POST -H "Host: pecan.localhost" \ | |
| # -F 'hostname=' \ | |
| # -F 'modelid=' \ | |
| # -F 'sitegroupid=' \ | |
| # -F 'siteid=' \ | |
| # -F 'sitename=' \ | |
| # -F 'pft[]=' \ | |
| # -F 'start=' \ | |
| # -F 'end=' \ | |
| # -F 'input_met=' \ | |
| # -F 'email=' \ | |
| # -F 'notes=' \ | |
| # 'http://172.17.0.1/pecan/04-runpecan.php' | |
| # Adjust the sleep time according to the model you are running | |
| # Append the workflowid and ID after `out/` by 1 for every consecutive run | |
| # curl -v -L -H "Host: pecan.localhost" \ | |
| # 'http://172.17.0.1/pecan/dataset.php?workflowid=(ID)&type=file&name=out/(ID)/sipnet.out' |
|
|
||
| curl -v -L -H "Host: pecan.localhost" \ | ||
| 'http://172.17.0.1/pecan/dataset.php?workflowid=99000000001&type=file&name=out/99000000001/sipnet.out' | ||
|
|
There was a problem hiding this comment.
Add a step here verifying that sipnet.out is free from errors / contains the expected output values?
|
Left some comments above focused on technical implementation, but from a broader perspective it's worth making sure we're aligned that "Whole-docker-stack Sipnet test via GitHub Actions every Monday" is a good approach. For my part I agree it's smart to make sure we have tests that exercise the whole docker-compose project, but given we already test Sipnet in the daily CI suite I don't have a good sense how much more information we get from testing it again inside the full Docker stack. Emphasis that I really do mean "I don't know", not "I think the proposed approach is wrong" Pinging @meetagrawal09 for thoughts on how this aligns/interacts with your planned CI development this summer |
|
@infotroph Some points that come to my mind when I check this implementation are :
|
|
Closing this PR in favour of #3345, please reopen if you disagree |
Description
In this PR, we have used the full docker stack to test SIPNET by using Docker Compose, this test will run weekly on Mondays.
Motivation and Context
From Discussions in #3184, It would be wise to have a script file and have our cURL POST requests for different models in it as the Matrix option would have to bring up the whole docker stack for each job of the Matrix.
In this PR, we have added the SIPNET model test, which outputs the sipnet.out (model output) file, Test results can be found in this workflow
In the script file, template has been provided to correctly input data into the cURL request so that in future, People can add more models in the script file against which tests can be performed.
Review Time Estimate
Types of changes
Checklist:
This PR closes #3184